Papers with knowledge editing method
Knowledge Editing of Large Language Models Unconstrained by Word Order (2024.acl-srw)
Copied to clipboard
| Challenge: | Existing methods for identifying knowledge neurons for large language models have been challenging for black-boxed models . a new method is proposed to edit the knowledge held by the LLMs . |
| Approach: | They propose a method that identifies the knowledge neurons that encode the target knowledge and adjusts the parameters associated with these neurons to update the knowledge. |
| Outcome: | The proposed method outperforms existing methods on English and Japanese . it eliminates word order constraints and allows flexible locating regardless of the language . |
AdaEdit: Advancing Continuous Knowledge Editing For Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Existing knowledge editing methods that can efficiently update knowledge in LLMs are limited due to budget constraints. |
| Approach: | They propose a method that can enhance the performance of edited LLMs in large-size continuous editing regimes. |
| Outcome: | Extensive empirical evaluations on multiple LLMs show that the proposed method outperforms existing methods without compromising the general abilities of these models. |
Interpretability-based Tailored Knowledge Editing in Transformers (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for modifying in-context learning fail to analyze the instability of in-constitu learning outcomes. |
| Approach: | They propose a model-based knowledge editing method that considers the unique information flow of each sample and aims to correct errors without costly retraining. |
| Outcome: | The proposed method exploits the critical role of feed-forward MLPs in decoder-only models and reveals diverse attribute recall across transformer layers, guiding edits to specific features at different depths and mitigating over-editing issues. |
Knowledge Decoupling via Orthogonal Projection for Lifelong Editing of Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for enhancing large language models (LLMs) have achieved some success, but their knowledge understanding and memory capacity significantly degrades after extensive editing. |
| Approach: | They propose a method that stores the basis vectors of the representation space of past edits in a knowledge cache and projects the gradient of the current edit onto a space orthogonal to previous knowledge for updating. |
| Outcome: | The proposed method improves question-answering ability and hallucination mitigation by 14% and 61% for large language models after 3,000 edits. |
KELE: Residual Knowledge Erasure for Enhanced Multi-hop Reasoning in Knowledge Editing (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing knowledge editing techniques show limitations when applied to multi-hop reasoning . residual single-hop knowledge causes edited models to revert to original answers . |
| Approach: | They propose a knowledge editing method that incorporates a Knowledge Erasure mechanism for Large language model Editing (KELE) they propose an erasure function for residual knowledge and an injection function for new knowledge . |
| Outcome: | The proposed method significantly improves multi-hop reasoning capability of edited models. |